fix: prevent multi-stage tests from passing when earlier stages fail#401
fix: prevent multi-stage tests from passing when earlier stages fail#401
Conversation
There was a problem hiding this comment.
Pull request overview
This PR updates the ESP-IDF Unity test runner integration to ensure multi-stage Unity tests correctly fail if any earlier stage fails (instead of potentially reporting success based only on later-stage output). It also adds a dedicated QEMU unit-test app fixture and adjusts the QEMU test expectations accordingly.
Changes:
- Update
IdfUnityDutMixinmulti-stage execution flow to collect and squash failures across sub-stages into a single failing JUnit testcase. - Add a new
unit_test_app_qemufixture (source + prebuilt artifacts) tailored for validating multi-stage behavior under QEMU. - Update QEMU pytest to use the new fixture and assert the corrected JUnit pass/fail counts.
Reviewed changes
Copilot reviewed 8 out of 11 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
pytest_embedded_idf/unity_tester.py |
Collect failures between multi-stage subcases and ensure earlier-stage failures are reflected in final results. |
pytest-embedded-qemu/tests/test_qemu.py |
Switches the test to the QEMU-specific unit-test fixture and updates expected JUnit totals. |
tests/fixtures/unit_test_app_qemu/** |
Adds a QEMU unit-test app fixture (including prebuilt binaries/config) containing normal + multi-stage pass/fail cases. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| for sub_case in case.subcases: | ||
| if sub_case != case.subcases[0]: | ||
| self._get_ready(timeout) | ||
| ready_before = self._get_ready(timeout, return_before=True) |
There was a problem hiding this comment.
What if there are multiple test cases, such as:
- multi-stage test 1
- multi-stage test 2
Will multi-stage test 2 have access to the before logs from multi-stage test 1? Is it possible that it could be affected by them?
There was a problem hiding this comment.
thank you. added ready_before = None as a init
There was a problem hiding this comment.
no. thought it over again. i don't think it would happen.
hfudev
force-pushed
the
fix/multi-stage-tests
branch
from
50767a6 to
7457ba1
Compare
|
No description provided.